[QNN EP] Add QuickGELU operator support for QNN provider #27034
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds support for the QuickGELU operator in the QNN provider: - Implement QuickGeluOpBuilder to handle QuickGELU operations - Add registration for QuickGELU in op_builder_factory - Add comprehensive tests for CPU and HTP backends - Support both float and quantized (QDQ) versions Implementation details: - QNN doesn't have a direct operator to map QuickGelu - QuickGELU is decomposed as x * sigmoid(alpha * x)
Contributor
Author
|
@edgchen1 and @yuslepukhin |
Contributor
|
/azp run Linux QNN CI Pipeline,Windows ARM64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Contributor
Author
|
@edgchen1 and @yuslepukhin |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds support for the QuickGELU operator in the QNN provider by decomposing it into basic operations: x * sigmoid(alpha * x). Since QNN doesn't have a direct QuickGELU operator, the implementation uses element-wise multiplication and sigmoid operations to enable running models with QuickGELU on HTP backend.
Changes:
- Implemented QuickGeluOpBuilder to decompose QuickGELU into basic QNN operations
- Registered QuickGELU operator in the QNN op builder factory
- Added comprehensive test coverage for CPU and HTP backends supporting float32, float16, and quantized (QDQ) modes
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| onnxruntime/core/providers/qnn/builder/opbuilder/quick_gelu_op_builder.cc | Implements QuickGeluOpBuilder that decomposes QuickGELU into multiplication and sigmoid operations, with optimization to skip alpha multiplication when alpha≈1.0 |
| onnxruntime/core/providers/qnn/builder/op_builder_factory.h | Declares CreateQuickGeluOpBuilder function for operator registration |
| onnxruntime/core/providers/qnn/builder/op_builder_factory.cc | Registers QuickGelu operator with the builder factory |
| onnxruntime/test/providers/qnn/quick_gelu_op_test.cc | Comprehensive test suite covering CPU/HTP backends with float32, float16, and QDQ quantized modes using various alpha values |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
onnxruntime/core/providers/qnn/builder/opbuilder/quick_gelu_op_builder.cc
Outdated
Show resolved
Hide resolved
…_builder.cc Co-authored-by: Copilot <[email protected]>
Contributor
Author
|
@edgchen1 and @yuslepukhin |
yuslepukhin
reviewed
Jan 16, 2026
onnxruntime/core/providers/qnn/builder/opbuilder/quick_gelu_op_builder.cc
Show resolved
Hide resolved
Contributor
|
/azp run Linux QNN CI Pipeline,Windows ARM64 QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows GPU Doc Gen CI Pipeline |
Member
|
/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
1 similar comment
|
Azure Pipelines successfully started running 4 pipeline(s). |
Member
|
/azp run Linux QNN CI Pipeline,Windows ARM64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 2 pipeline(s). |
adrianlizarraga
approved these changes
Jan 16, 2026
edgchen1
approved these changes
Jan 16, 2026
alex-spacemit
pushed a commit
to spacemit-com/onnxruntime
that referenced
this pull request
Jan 20, 2026
…7034) ### Description Add support for the QuickGELU operator in the QNN provider: - Implement QuickGeluOpBuilder to handle QuickGELU operations - Add registration for QuickGELU in op_builder_factory - Add comprehensive tests for CPU and HTP backends - Support both float and quantized (QDQ) versions ### Motivation and Context - QNN doesn't have a direct operator to map QuickGelu so decompose it as x * sigmoid(alpha * x) for computing the whole model on HTP to improve inference time. --------- Co-authored-by: Copilot <[email protected]>
tianleiwu
pushed a commit
that referenced
this pull request
Jan 21, 2026
### Description Add support for the QuickGELU operator in the QNN provider: - Implement QuickGeluOpBuilder to handle QuickGELU operations - Add registration for QuickGELU in op_builder_factory - Add comprehensive tests for CPU and HTP backends - Support both float and quantized (QDQ) versions ### Motivation and Context - QNN doesn't have a direct operator to map QuickGelu so decompose it as x * sigmoid(alpha * x) for computing the whole model on HTP to improve inference time. --------- Co-authored-by: Copilot <[email protected]> (cherry picked from commit 9d108d0)
This was referenced Jan 21, 2026
tianleiwu
added a commit
that referenced
this pull request
Jan 23, 2026
### Description This PR cherry-picks the following changes for the 1.24.0 release. ### Cherry-picked Commits | Commit | Commit Title | Author | |---|---|---| | 744e7fe | Add type definitions, registration, utilities for INT2/UINT2 support (#26824) | vraspar | | 530a1fb | [QNN EP] Add BFloat16 dtype support in QNN EP (#26987) | tirupath-qti | | 8e050d1 | Implement new experimental lookup-based matrix multiplication method(TMAC) (#26695) | vraspar | | 2d2ba6b | [MLAS/CPU EP] Improve performance of Silu activation path within the QuickGelu CPU kernel (#26753) | Hariharan Seshadri | | 1c02b79 | [QNN EP] Add support for handling 0-dimension for Concat Op (#27000) | Ashwath Shankarnarayan | | cc2b01b | Fix ClipQuantFusion crash when Clip has multiple input edges (#27016) | Edward Chen | | bbd3850 | [QNN EP] Support quantized BatchNorm with per-channel DQ params on QNN HTP (#26959) | qti-yuduo | | d8f0318 | Add API to get ep graph partitioning info (#26781) | Adrian Lizarraga | | b912b18 | [OVEP] OpenVINO EP Features and bug-fixes for ORT-1.24 - Follow up (#27007) | Preetha Veeramalai | | ba11af4 | [QNN-EP] Add MatMulNBits translation for GPU (#26340) | quic-tirupath | | c03c419 | [MLAS/NEON] Add dedicated kernel for depthwise convolution for ARM64 using NEON intrinsics (#26688) | Hariharan Seshadri | | e7dfd69 | [QNN-EP] Support alternate Layernorm fusion pattern in QNN preprocess (#26060) | qti-mattsinc | | 4013dc1 | Implement multithreading in qgemm_kleidi (#26301) | Melike Kaptan | | 9f06181 | [CXX] Enable users to specify custom OrtSyncStream via RunOptions (#26988) | Dmitri Smirnov | | cfccd64 | Added support for QMX kernels in MLAS (#26849) | qti-vaiskv | | 29d9b2f | Tweak external resource importer handle structs (#27040) | Scott McKay | | 9d108d0 | [QNN EP] Add QuickGELU operator support for QNN provider (#27034) | tirupath-qti | | b35688f | Add INT2 and UINT2 support for QDQ, transpose and cast ops (#27022) | vraspar | | 6d34aba | Introducing BF16 Pointwise NCHWc Convolution for Arm64 (#26838) | Rohanjames1997 | | 36017ad | [EP ABI] Add CreateCustomOpDomains() API for plugin EP to register custom ops (#27050) | Chi Lo | | 50a03e4 | Add a new pipeline for CUDA 13 nuget builds (#27023) | eserscor | | a0d4439 | [EP ABI] Update Graph_GetGraphView() implementation (#26711) | Chi Lo | | 34bb209 | [webgpu] Fix a bug for im2col (#27069) | Wenqin Yang | | 46e8d45 | [QNN EP] Add FusedMatMul operator support (#27044) | tirupath-qti | | 5e7e7a3 | Disable Float32_2Bits_Asymmetric_256x256 test (#27046) | vraspar | | 39f966e | Fix Doxygen documentation build error in onnxruntime_c_api.h (#27083) | Nick Eubanks | | 8a7a797 | Print tensor for new packed type of 2 bits (#27064) | Tianlei Wu | | 01f40e6 | Fix GPU JAR testing on Linux (#27011) | eserscor | | b6ed7f3 | Fix warning around ununsed code in QNN Android Emulator builds by clang (#27026) | Hariharan Seshadri | | d7daa45 | Raise the timeout for the ios simulator job (#27045) | Hariharan Seshadri | | 7e1d818 | upgrade emsdk to 4.0.23 (#27029) | Yulong Wang | | 347b990 | Fix failing mainline build on Arm64 linux (#27101) | Rohanjames1997 | | f481b17 | Add dedicated API to support extracting compatibility string from model metadata (#27015) | adrastogi | --------- Signed-off-by: Liqun Fu <[email protected]> Signed-off-by: bfilipek <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Jonathan Clohessy <[email protected]> Signed-off-by: Christian Bourjau <[email protected]> Signed-off-by: melkap01 <[email protected]> Co-authored-by: vraspar <[email protected]> Co-authored-by: tirupath-qti <[email protected]> Co-authored-by: Ashwath Shankarnarayan <[email protected]> Co-authored-by: Liqun Fu <[email protected]> Co-authored-by: carzh <[email protected]> Co-authored-by: Hector Li <[email protected]> Co-authored-by: carzh <[email protected]> Co-authored-by: Vrajang Parikh <[email protected]> Co-authored-by: Hariharan Seshadri <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Edward Chen <[email protected]> Co-authored-by: Yuduo Wu <[email protected]> Co-authored-by: Adrian Lizarraga <[email protected]> Co-authored-by: Preetha Veeramalai <[email protected]> Co-authored-by: jatinwadhwa921 <[email protected]> Co-authored-by: jatinwadhwa921 <[email protected]> Co-authored-by: saurabh <[email protected]> Co-authored-by: Ankit Maheshkar <[email protected]> Co-authored-by: sfatimar <[email protected]> Co-authored-by: Javier Martinez <[email protected]> Co-authored-by: Bartlomiej Filipek <[email protected]> Co-authored-by: bopeng1234 <[email protected]> Co-authored-by: Eric Crawford <[email protected]> Co-authored-by: MayureshV1 <[email protected]> Co-authored-by: TejalKhade28 <[email protected]> Co-authored-by: Vishnudas Thaniel S <[email protected]> Co-authored-by: Yaru Du <[email protected]> Co-authored-by: Ryan Metcalfe <[email protected]> Co-authored-by: Dvoretckii, Mikhail <[email protected]> Co-authored-by: Pallavi Gupta <[email protected]> Co-authored-by: Jianhui Dai <[email protected]> Co-authored-by: Jiajia Qin <[email protected]> Co-authored-by: Changming Sun <[email protected]> Co-authored-by: Fei Chen <[email protected]> Co-authored-by: Yulong Wang <[email protected]> Co-authored-by: Akupadhye <[email protected]> Co-authored-by: Wang Ning <[email protected]> Co-authored-by: Maximilian Müller <[email protected]> Co-authored-by: Chi Lo <[email protected]> Co-authored-by: George Wu <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Wanming Lin <[email protected]> Co-authored-by: quic-calvnguy <[email protected]> Co-authored-by: Jie Chen <[email protected]> Co-authored-by: xhcao <[email protected]> Co-authored-by: Wei-Sheng Chin <[email protected]> Co-authored-by: quic-hungjuiw <[email protected]> Co-authored-by: Ian Hunter <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kunal-vaishnavi <[email protected]> Co-authored-by: Jeff Kilpatrick <[email protected]> Co-authored-by: Jeff Kilpatrick <[email protected]> Co-authored-by: Scott McKay <[email protected]> Co-authored-by: Nenad Banfic <[email protected]> Co-authored-by: derdeljan-msft <[email protected]> Co-authored-by: n1harika <[email protected]> Co-authored-by: Ryan Metcalfe <[email protected]> Co-authored-by: Jaswanth Gannamaneni <[email protected]> Co-authored-by: Klimenko, Mikhail <[email protected]> Co-authored-by: liang <[email protected]> Co-authored-by: Garth Long <[email protected]> Co-authored-by: Jonathan Clohessy <[email protected]> Co-authored-by: Akshay Sonawane <[email protected]> Co-authored-by: Christopher Warrington <[email protected]> Co-authored-by: Ishwar Raut <[email protected]> Co-authored-by: Gaurav Garg <[email protected]> Co-authored-by: Xinpeng Dou <[email protected]> Co-authored-by: adrastogi <[email protected]> Co-authored-by: Aditya Rastogi <[email protected]> Co-authored-by: qti-hungjuiw <[email protected]> Co-authored-by: Pradeep Sakhamoori <[email protected]> Co-authored-by: Adam Pocock <[email protected]> Co-authored-by: mingyue <[email protected]> Co-authored-by: Susanta Bhattacharjee <[email protected]> Co-authored-by: Jozef Wludzik <[email protected]> Co-authored-by: Rajeev Sekar <[email protected]> Co-authored-by: Mayuresh M Varerkar <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Wenqin Yang <[email protected]> Co-authored-by: xieofxie <[email protected]> Co-authored-by: hualxie <[email protected]> Co-authored-by: Joshua Lochner <[email protected]> Co-authored-by: Christian Bourjau <[email protected]> Co-authored-by: Xiaofei Han <[email protected]> Co-authored-by: Dmitri Smirnov <[email protected]> Co-authored-by: chunghow-qti <[email protected]> Co-authored-by: Guenther Schmuelling <[email protected]> Co-authored-by: Jiawei Shao <[email protected]> Co-authored-by: czekun <[email protected]> Co-authored-by: Jaskaran Singh Nagi <[email protected]> Co-authored-by: quic-tirupath <[email protected]> Co-authored-by: qti-mattsinc <[email protected]> Co-authored-by: Melike Kaptan <[email protected]> Co-authored-by: Damien Dooley <[email protected]> Co-authored-by: qti-vaiskv <[email protected]> Co-authored-by: Rohanjames1997 <[email protected]> Co-authored-by: eserscor <[email protected]> Co-authored-by: eserscor <[email protected]> Co-authored-by: Nick Eubanks <[email protected]> Co-authored-by: adrastogi <[email protected]> Co-authored-by: Rohanjames1997 <[email protected]>
alex-spacemit
pushed a commit
to spacemit-com/onnxruntime
that referenced
this pull request
Jan 27, 2026
…7034) Add support for the QuickGELU operator in the QNN provider: - Implement QuickGeluOpBuilder to handle QuickGELU operations - Add registration for QuickGELU in op_builder_factory - Add comprehensive tests for CPU and HTP backends - Support both float and quantized (QDQ) versions - QNN doesn't have a direct operator to map QuickGelu so decompose it as x * sigmoid(alpha * x) for computing the whole model on HTP to improve inference time. --------- Co-authored-by: Copilot <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add support for the QuickGELU operator in the QNN provider:
Motivation and Context